Reviewing user-created content before website presentation

ABSTRACT

Methods and systems for reviewing content items created by users for presentation in association with a website are provided. Automatic scrubbing (followed by manual review, if necessary) of user-created content items that have yet to be presented in association with a website is provided as is manual review of user-created content items that have already been presented in association with a website at the request of a user.

CROSS-REFERENCE TO RELATED APPLICATIONS

Not Applicable.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable.

BACKGROUND

Websites that implement community features provide users the ability topresent content in association therewith for consumption by all users.Examples of this user-created content are product reviews and discussiongroup posts. Websites typically contain a web page with guidelines thatusers are asked to adhere to when submitting content for presentation.Some users, either intentionally or unintentionally, violate publishedguidelines by submitting content for presentation that is abusive due toprofanity, spam, or inclusion of other content that violates thewebsite's guidelines for user-created content. Presentation of thisabusive content on the website can lead to customer dissatisfaction,distrust, or even legal issues if the content is egregious enough (e.g.,if the user-created content includes threats, racist language, or thelike).

Existing processes for detecting and preventing publication of abusiveuser-created content are either manual processes that involve a largenumber of support staff reading content in its entirety, or automaticprocesses with overly simplistic logic that users can easily circumventby slightly obfuscating potentially abusive language.

BRIEF SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

In embodiments, the present invention provides systems and methods forautomatically reviewing content items created by users prior topresentation of the content items in association with a website.Utilizing such systems and methods, user-created content that containsone or more objectionable words/phrases is prevented from beingpresented in association with the website. Additionally, embodiments ofthe present invention provide systems and methods for permitting usersvisiting a website having user-created content presented in associationtherewith to submit content items already presented in association withthe website for review. A user may choose to do this, for instance, ifs/he believes a particular content item violates the website's publishedguidelines relating to profanity, spam, or the like. In still furtherembodiments, the present invention provides systems and methods forpermitting an administrative user to access stored suspect contentitems, manually review them, and assign each one an objectionable ornon-objectionable status. If the content item is manually assigned anobjectionable status, it will either not be presented in associationwith the website or will be removed there from, whichever isappropriate. If the content item is manually assigned anon-objectionable status, it will either be presented in associationwith the website or will be permitted to remain presented in associationwith the website, whichever is appropriate.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The present invention is described in detail below with reference to theattached drawing figures, wherein:

FIG. 1 is a block diagram of an exemplary computing environment suitablefor use in implementing embodiments of the present invention;

FIG. 2 is a block diagram of an exemplary system architecture for use inimplementing embodiments of the present invention;

FIG. 3 is a flow diagram, in accordance with an embodiment of thepresent invention, illustrating a method for automatically reviewinguser-created content items prior to presentation in association with awebsite;

FIG. 4 is a flow diagram illustrating a method for accepting userrequests for manual review of content items already presented inassociation with a website, in accordance with an embodiment of thepresent invention;

FIG. 5 is a flow diagram illustrating a method, in accordance with anembodiment of the present invention, for reviewing suspect user-createdcontent items;

FIG. 6 is a screen display of an exemplary user interface for presentingsuspect user-created content items with the word/phrase whichprecipitated the assigning of a suspect status being singled out foreasier and more efficient review, in accordance with an embodiment ofthe present invention;

FIG. 7 is a screen display of an exemplary user interface for presentingan already published user-created content item which has been submittedfor review by a user visiting a website, in accordance with anembodiment of the present invention; and

FIG. 8 is a screen display of an exemplary user interface illustrating aviolation report submitted by a user in conjunction with a user-createdcontent item s/he has submitted for review, in accordance with anembodiment of the present invention.

DETAILED DESCRIPTION

The subject matter of the present invention is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of this patent.Rather, the inventors have contemplated that the claimed subject mattermight also be embodied in other ways, to include different steps orcombinations of steps similar to the ones described in this document, inconjunction with other present or future technologies. Moreover,although the terms “step” and/or “block” may be used herein to connotedifferent elements of methods employed, the terms should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

Embodiments of the present invention provide systems and methods forautomatically reviewing content items created by users prior topresentation of the content items in association with a website.Utilizing embodiments of the systems and methods herein disclosed,user-created content that contains one or more objectionable words orphrases is prevented from being presented in association with thewebsite. Additional embodiments of the present invention provide systemsand methods for permitting users visiting a website having user-createdcontent presented in association therewith to submit content itemsalready presented in association with the website for review. A user maychoose to do this, for instance, if s/he believes a particular contentitem violates the website's published guidelines relating to profanity,spam, or the like. In still further embodiments, the present inventionprovides systems and methods for storing suspect content items, i.e.,content items assigned a suspect status either by a previous automaticreview process prior to presentation in association with the website orcontent items already presented in association with the website that auser has submitted for review, for manual review. An administrative usermay then access the stored suspect content items, review them, andassign each one an objectionable or non-objectionable status. If thecontent item is manually assigned an objectionable status, it willeither not be presented in association with the website or will beremoved there from, whichever is appropriate. If the content item ismanually assigned a non-objectionable status, it will either bepresented in association with the website or will be permitted to remainpresented in association with the website, whichever is appropriate.

Having briefly described an overview of the present invention, anexemplary operating environment for embodiments of the present inventionis described below.

Referring initially to FIG. 1 in particular, an exemplary operatingenvironment for implementing the present invention is shown anddesignated generally as computing device 100. Computing device 100 isbut one example of a suitable computing environment and is not intendedto suggest any limitation as to the scope of use or functionality of theinvention. Neither should the computing-environment 100 be interpretedas having any dependency or requirement relating to any one orcombination of components illustrated.

The invention may be described in the general context of computer codeor machine-useable instructions, including computer-executableinstructions such as program modules, being executed by a computer orother machine, such as a personal data assistant or other handhelddevice. Generally, program modules including routines, programs,objects, components, data structures, and the like, refer to code thatperform particular tasks or implement particular abstract data types.The invention may be practiced in a variety of system configurations,including hand-held devices, consumer electronics, general-purposecomputers, more specialty computing devices, etc. The invention may alsobe practiced in distributed computing environments where tasks areperformed by remote-processing devices that are linked through acommunications network.

With continued reference to FIG. 1, computing device 100 includes a bus110 that directly or indirectly couples the following devices: memory112, one or more processors 114, one or more presentation components116, input/output ports 118, input/output components 120, and anillustrative power supply 122. Bus 110 represents what may be one ormore busses (such as an address bus, data bus, or combination thereof).Although the various blocks of FIG. 1 are shown with lines for the sakeof clarity, in reality, delineating various components is not so clear,and metaphorically, the lines would more accurately be grey and fuzzy.For example, one may consider a presentation component such as a displaydevice to be an I/O component. Also, processors have memory. Werecognize that such is the nature of the art, and reiterate that thediagram of FIG. 1 is merely illustrative of an exemplary computingdevice that can be used in connection with one or more embodiments ofthe present invention. Distinction is not made between such categoriesas “workstation,” “server,” “laptop,” “hand-held device,” etc., as allare contemplated within the scope of FIG. 1 and reference to “computingdevice.”

Computing device 100 typically includes a variety of computer-readablemedia. By way of example, and not limitation, computer-readable mediamay comprises Random Access Memory (RAM); Read Only Memory (ROM);Electronically Erasable Programmable Read Only Memory (EEPROM); flashmemory or other memory technologies; CDROM, digital versatile disks(DVD) or other optical or holographic media; magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,carrier wave or any other medium that can be used to encode desiredinformation and be accessed by computing device 100.

Memory 112 includes computer-storage media in the form of volatileand/or nonvolatile memory. The memory may be removable, nonremovable, ora combination thereof. Exemplary hardware devices include solid-statememory, hard drives, optical-disc drives, etc. Computing device 100includes one or more processors that read data from various entitiessuch as memory 112 or I/O components 120. Presentation component(s) 116present data indications to a user or other device. Exemplarypresentation components include a display device, speaker, printingcomponent, vibrating component, etc.

I/O ports 118 allow computing device 100 to be logically coupled toother devices including I/O components 120, some of which may be builtin. Illustrative components include a microphone, joystick, game pad,satellite dish, scanner, printer, wireless device, etc.

Referring now to FIG. 2, a block diagram is illustrated that shows anoverall system architecture 200 for review of content items created byusers for presentation in association with a website. It will beunderstood and appreciated by those of ordinary skill in the art thatthe overall system architecture 200 shown in FIG. 2 is merely an exampleof one suitable computing environment and is not intended to suggest anylimitation as to the scope of use or functionality of the presentinvention. Neither should the overall system architecture 200 beinterpreted as having any dependency or requirement related to anysingle component or combination of components illustrated therein.

System 200 includes a user device 202 connected to an administrativedevice 204 via a network 206. Each of the user device 202 and theadministrative device 204 shown in FIG. 2 may be any type of computingdevice, such as, for example, computing device 100 described above withreference to FIG. 1. By way of example only and not limitation, the userdevice 202 and/or the administrative device 204 may be a personalcomputer, desktop computer, laptop computer, handheld device, consumerelectronic device, and the like. It should be noted, however, that theinvention is not limited to implementation on such computing devices,but may be implemented on any of a variety of different types ofcomputing devices within the scope of embodiments of the presentinvention.

As shown in FIG. 2, the user device 202 includes a content itemreceiving component 208, an accessing component 210, a comparingcomponent 212, and a presentation component 214 for implementingembodiments of the present invention. In some embodiments, thecomponents 208, 210, 212 and 214 may be implemented as stand-aloneapplications. In other embodiments, the components 208, 210, 212 and 214may be integrated directly into the operating system for the user device202. It will be understood by those skilled in the art that thecomponents 208, 210, 212 and 214 illustrated in FIG. 2 are exemplary innature and in number and should not be construed as limiting. Any numberof components may be employed to achieve the desired functionalitywithin the scope of embodiments of the present invention.

The content item receiving component 208 is configured for receiving oneor more content items input by a user at the user device 202 forpresentation in association with a website, each content item having atleast one word or phrase included therein. In one embodiment, thecontent item receiving component 208 is further configured fortransmitting received content items to a content item database 216 forstorage (via network 206). By way of example only, a content item inputby the user may be a product review, discussion group posting, or thelike.

The accessing component 210 of the user device 202 is configured foraccessing user-created content items (e.g., from the content itemdatabase 216) and one or more filter lists, e.g., from a filter listdatabase 218, via network 206. Filter lists are discussed more fullyherein below and typically include one or more words or phrases having apre-determined objectionable or suspect status associated therewith.

The comparing component 212 is configured for comparing the words orphrases included in user-created content items with the words or phrasesincluded in one or more filter lists. In one embodiment, the comparingcomponent 212 is further configured for determining if any matches occurbetween the words or phrases included in a user-created content item andthe words or phrases included in one or more filter lists. Stillfurther, in one embodiment, the comparing component 212 is configuredfor transmitting the results of such determinations (via network 206)for storage in association with the appropriate user-created contentitems. By way of example only, the comparing component 212 may transmitthe results of such determination to content item database 216 forstorage in association with the user-created content items to which suchdeterminations relate. In one embodiment, the comparing component 212 isadditionally configured for determining a status (e.g., objectionable,non-objectionable, or suspect) for user-created content items based upona status of any matching words or phrases included therein. In oneembodiment, the comparing component 212 is further configured fortransmitting such status determinations to content item database 216 forstorage in association with the user-created content items to which suchdeterminations pertain. Assigning of status to words/phrases and contentitems is more fully described below with reference to FIG. 3.

The presentation component 214 of the user device 202 is configured topresent non-objectionable, user-created content in association with awebsite. Typically, presentation of the non-objectionable, user-createdcontent comprises displaying the user-created content on a displaydevice associated with the user device 202. However, other types ofpresentation, such as an audible presentation, may also be providedwithin the scope of embodiments of the present invention. In oneembodiment, the presentation component 214 of the user device 202 isfurther configured to present, in association with the website, a userinterface for receiving a user request for manual review of user-createdcontent items that have already been presented in association with awebsite and for transmitting such user requests for storage pending suchmanual review. In one embodiment, the user requests may be transmittedto the content item database 216 (via network 206) for storage.

Administrative device 204 includes an accessing component 220, apresentation component 222, and a determination receiving component 224for implementing embodiments of the present invention. In someembodiments, the components 220, 222 and 224 may be implemented asstand-alone applications. In other embodiments, the components 220, 222and 224 may be integrated directly into the operating system for theadministrative device 204. It will be understood by those skilled in theart that the components 220, 222 and 224 illustrated in FIG. 2 areexemplary in nature and in number and should not be construed aslimiting. Any number of components may be employed to achieve thedesired functionality within the scope of embodiments of the presentinvention.

The accessing component 220 of the administrative device 204 isconfigured to access user-created content items that have been assigneda suspect status. In one embodiment, the accessing component 220 mayaccess such user-created content items from the content item database216 (via network 206). As more fully described below, user-createdcontent items may be assigned a suspect status through an automaticreview process prior to presentation of the content item in associationwith the website or through a user submitting a content item that hasalready been presented in association with the website for manualreview.

The presentation component 222 of the administrative device 204 isconfigured to present suspect user-created items for manual review by anadministrative user. Typically, presentation of the suspect user-createditems comprises displaying the user-created content items on a displaydevice associated with the administrative device 204. However, othertypes of presentation, such as an audible presentation, may also beprovided within the scope of embodiments of the invention. In oneembodiment, the presentation component 222 is further configured topresent the suspect user-created items with the suspect word(s) orphrase(s) singled out (e.g., highlighted, underlined, bolded,italicized, colored, or the like) for easier and more efficient manualreview.

The determination receiving component 224 is configured for receiving adetermination from an administrative user regarding whether a suspectuser-created content item is to be assigned an objectionable ornon-objectionable status subsequent to manual review. In one embodiment,the determination receiving component 224 is further configured totransmit such determination for storage in association with theappropriate user-created content item. In one embodiment, thedetermination may be transmitted to the content item database 216 (vianetwork 206) for storage in association with the user-created contentitem to which it pertains.

Referring now to FIG. 3, a flow diagram is illustrated that shows anexemplary method 300 for automatically reviewing a user-created contentitem prior to presentation on a website. Initially, as indicated atblock 310, a content item created by a user that has been submitted forpresentation in association with a website is received, e.g., by contentitem receiving component 208 of FIG. 2, as is an indication of suchreceipt. By way of example only and not limitation, the user-createdcontent item may be a product review, discussion group posting, or thelike and has at least one word or phrase included therein. In oneembodiment, upon receipt of the user-created content item, such contentitem is assigned a status of null and is transmitted (via network 206)to a content item database 216 for storage. If desired, the user-createdcontent item may be stored in a field of the content item database 216reserved for storage of content items that have yet to undergosubstantive content review (i.e., content items having a null status).

Upon receipt of the indication that a user-created content item has beenreceived, or based upon a pre-determined time interval, a content review(or scrubbing) process is automatically initiated, that is, without userintervention. Initially, the user-created content item is accessed,e.g., by accessing component 210 of FIG. 2, as indicated at block 312.Prior to, subsequent to, or simultaneously with accessing of theuser-created content item, one or more filter lists containing wordsand/or phrases having an objectionable or suspect status associatedtherewith is accessed, as indicated at block 314. As used herein, theterm “filter list” refers to a list of words and/or phrases that havepreviously been determined to have an objectionable or suspect statusassociated therewith. As such, each filter list includes not only theobjectionable or suspect words/phrases but also a status associated witheach word/phrase. In one embodiment, those words/phrases that areassociated with an objectionable status will also include acategorization indicative of a level of abusiveness associated with theobjectionable status. For instance, one word/phrase may be associatedwith an objectionable-hate status which may be accorded a higher levelof abusiveness than another word/phrase that is associated with anobjectionable-profanity status. Other examples of categories that may beassociated with objectionable words/phrase include, without limitation,spam, garbage, and personal information. Generally, words/phrases havinga suspect status are viewed as less abusive than any word/phrase havingan objectionable status associated therewith.

In one embodiment, a single filter list may be accessed for comparisonof the words/phrases included therein with the words/phrase included inthe user-created content item being reviewed. In another embodiment,multiple filter lists may be accessed for comparison. For instance, aglobal filter list that includes a list of filter words/phrases that areapplicable to user-created content items to be presented on any websitemay be accessed in addition to a website-specific filter list thatincludes a list of filter words/phrases that are applicable only touser-created content items to be presented in association with theparticular website to which the website-specific filter list isdirected.

Subsequently, as indicated at block 316, the words and/or phrasesincluded in the one or more filter lists are compared (e.g., utilizingcomparing component 212 of the user device 202 of FIG. 2) with the wordsand/or phrases included in the user-created content item. Based uponsuch comparison, it is determined whether one or more words/phrases fromthe filter list(s) exist in the user-created content item. This isindicated at block 318. Typically, if multiple filter lists areaccessed, the filter lists will be assigned an order for comparison. Forinstance, if a global filter list and a website-specific filter list areaccessed, the global filter list may be utilized for comparison firstand upon completion of comparison of the words/phrases included in theglobal filter list with the words/phrases included in the user-createdcontent item, the website-specific filter list may be utilized forcomparison. Typically, discrepancies between the multiple filter listswill be decided in favor of only one of the filter lists. In oneembodiment, a global filter list and a website specific filter list areaccessed and any discrepancies that arise are decided in favor of thewebsite-specific filter list.

Comparisons may be made between the words/phrases of the filter list(s)with the words/phrases of the user-created content item being reviewedutilizing exact and/or pattern matching techniques. Although notuniversally the case, words/phrases in the filter list(s) associatedwith an objectionable status may be matched based upon an exact matchingtechnique and words/phrases in the filter list(s) associated with asuspect status may be matched based upon a pattern matching technique.Various exact and pattern matching techniques are known to those ofordinary skill in the art and, accordingly, are not further discussedherein. Typically, comparisons are made on a word/phrase by word/phrasebasis, although embodiments involving comparison of predetermined fieldsand/or the content item as a whole are contemplated to be within thescope of embodiments hereof.

If none of the words/phrases from the filter list(s) exist in theuser-created content item, the content item is assigned a status ofnon-objectionable or okay (that is, okay to publish) and presented onthe website, as indicated at block 320. In one embodiment, suchpresentation is performed by the presentation component 214 of the userdevice 202 of FIG. 2. However, as indicated at block 322, if it isdetermined at block 318 that one or more words/phrases from the filterlist(s) exist in the user-created content item, the content item isassigned the status of the most abusive word/phrase found therein. Thatis, if a single word/phrase included in the user-created content itembeing reviewed matches a single word/phrase included in the filterlist(s), the status of the matching word/phrase is assigned to theuser-created content item. However, if multiple words/phrases includedin the user-created content item being reviewed match multiplewords/phrases included in the filter list(s) to which such content itemis compared, the content item as a whole is assigned the status of thematching word/phrase having the highest level of abusiveness associatedtherewith.

Subsequently, it is determined whether the most abusive word/phrasefound in the user-created content item is truly objectionable (that is,has an objectionable status associated therewith) or merely suspect(i.e., has a suspect status associated therewith), as indicated at block324. If the most abusive word/phase found in the user-created contentitem is truly objectionable, the user-created content item, theobjectionable word/phrase from the filter list, and the status,including any categorization associated therewith (e.g.,objectionable-hate), are stored in association with one another, asindicated at block 326. In one embodiment, the content item, theobjectionable word/phrase from the filter list, and the status arestored, e.g., in the content item database 216 of FIG. 2, in a fieldreserved for user-created content items that have been reviewed and forwhich no further action is required. Such storage is primarily for auditpurposes. In such situation, the user-created content item is notpresented in association with the website, as indicated at block 328.

If, however, it is determined at block 324 that the most abusiveword/phrase found in the user-created content item is merely suspect,the user-created content item, the suspect word/phrase from the filterlist(s), and the status (i.e., suspect) are stored in association withone another for further review. This is indicated at block 330. In oneembodiment, the content item, the suspect word/phrase from the filterlist, and the status are stored, e.g., in the content item database 216of FIG. 2, in a field reserved for user-created content items that havebeen reviewed and for which further action is required. In suchsituations, the user-created content is not presented in associationwith the website, as indicated at block 328, until further review iscompleted. One embodiment for implementing such further review is amanual review process discussed more fully herein below with referenceto FIG. 5.

With reference now to FIG. 4, a flow diagram is illustrated which showsa method 400 for accepting user requests for review of content itemsalready presented in association with a website. Initially, as indicatedat block 410, a request is received from a user at the website forreview of a user-created content item presented in association with thewebsite that s/he believes to be suspect. This is indicated at block410. A user may request review of a particular content item, forinstance, if s/he believes the content item violates the website'spublished guidelines relating to profanity, spam, or the like.Subsequently, as indicated at block 412, a user interface is presentedin association with the website configured to receive user inputregarding the user-created content item s/he believes is suspect and inneed of review.

Next, as indicated at block 414, an indication that the review requestsubmission form is complete is received and the suspect user-createdcontent item is stored in association with the completed submission formpending further review. This is indicated at block 416. In oneembodiment, the submission form and the user-created content item towhich it pertains are stored in the content item database 216 of FIG. 2in a field reserved for user-created content items assigned a suspectstatus by a user which are pending further review. In one embodiment,the user-created content item remains presented in association with thewebsite until further review is completed. However, embodiments of thepresent invention also contemplate removing the user-created contentitem from the website pending completion of further review.

With reference now to FIG. 5, a flow diagram is illustrated which showsa method 500 for reviewing suspect user-created content items.Initially, as indicated at block 510, a user-created content item havinga suspect status is accessed, e.g., from content item database 216utilizing accessing component 220 of administrative device 204 of FIG.2. Typically, access is initiated by an administrative user logging inand accessing a first-in, first-out queue of suspect content items.User-created content items may have a suspect status associatedtherewith based upon an automatic content review process of user-createdcontent that has not yet been presented in association with a website(for instance, the automatic content review process describedhereinabove with reference to FIG. 3) or based upon a user submitting analready presented user-created content item for review, e.g., utilizingthe submission process described hereinabove with reference to FIG. 4.

Subsequently, as indicated at block 512, a user interface is presented(e.g., utilizing presentation component 222 of administrative device 204of FIG. 2) that includes the suspect user-created content item (or atleast a portion thereof) with the word(s)/phrase(s) that precipitatedthe assigning of the suspect status singled out for easier and moreefficient review, if possible. An exemplary user interface 600illustrating such presentation is shown in the screen display of FIG. 6.In the illustrated embodiment, the word(s)/phrase(s) that precipitatedthe assigning of the suspect status is singled out by having the suspectstatus indicator displayed parenthetically in association with thesuspect word(s)/phrase(s) 610. It will be understood and appreciated bythose of ordinary skill in the art, however, that word(s)/phrase(s) maybe singled out using a variety of methods including, but not limited to,highlighting, underlining, bolding, italicizing, coloring, or the like.

In one embodiment, if the content item being reviewed is a content itemwhich has already been published in association with a website and hashad a visitor to the website submit it for review, the user interface700 illustrated in FIG. 7 may be presented. Note that in this instance,the word(s)/phrase(s) that precipitated the assigning of a suspectstatus is not singled out but that a link 710 to a violation report isprovided. Upon selection of the link 710, the administrative user willbe presented with the user interface 800 of FIG. 8 where s/he will beable to view the reason(s) for the website visitor's submission of thecontent item for review.

Subsequently, it is determined whether an objectionable word/phraseexists in the suspect user-created item presented, as indicated at block514. Typically, such determination is made by an administrative user whois manually reviewing the suspect user-created content item. In oneembodiment, if the word(s)/phrase(s) that precipitated the assigning ofthe suspect status is singled out, a determination is made whether thesingled out word/phrase should be assigned an objectionable ornon-objectionable status. In the embodiment illustrated in FIG. 6, theadministrative user may change the status of the word/phrase fromsuspect to objectionable or non-objectionable (okay) by selecting from amenu of options presented upon selection of the arrow 620. Uponcompletion of such manual review, an indication is received that themanual review is complete and that an objectionable/non-objectionabledetermination has been made, e.g., by user selection of the ‘save’indicator 630. In one embodiment, such indication, and the correspondingdetermination, are received by determination receiving component 224 ofthe administrative device 204 of FIG. 2.

In one embodiment, the administrative user may escalate the user-createdcontent item to a second tier of support for disposition is s/he cannotinitially assign the content item an objectionable or non-objectionablestatus with a high degree of confidence.

If it is determined at block 514 that an objectionable word/phrase doesnot exist in the suspect user-created item being reviewed, theuser-created content item is presented in association with the website,as indicated at block 516. Note that if the suspect user-created contentitem being reviewed is a content item that a user at the websitesubmitted for review, the user-created content item may already bepresented in association with the website. In this instance, thepresentation of the content item is permitted to persist withoutinterruption.

If, however, it is determined at block 514 that an objectionableword/phrase does exist in the suspect user-created content item beingreviewed, the user-created content item is assigned the status of themost abusive word/phrase found therein. This is indicated at block 518.That is, if the administrative user determines that the suspectuser-created content item contains a word/phrase that should be assignedan objectionable-hate status and a word/phrase that should be assignedan objectionable-profanity status, the suspect user-created content itemis assigned the status objectionable-hate, provided theobjectionable-hate status has a higher level of abusiveness than theobjectionable-profanity status.

Next, as indicated at block 520, the user-created content item, thefilter word/phrase that precipitated the assigning of an objectionablestatus, and the assigned status (including any categorization, ifapplicable) are stored, e.g., in content item database 216 of FIG. 2. Inone embodiment, such user-created content item, the offendingword/phrase, and the status are stored in a field reserved foruser-created content items that have been reviewed and for which nofurther action is required. Such storage is primarily for auditpurposes. In such situation, the user-created content item is notpresented in association with the website, as indicated at block 522.Note that if the suspect user-created content item being reviewed is acontent item that a user at the website submitted for review, theuser-created content item may already be presented in association withthe website. In this instance, the presentation of the content item isterminated.

Whether the user-created content item is determined to contain anobjectionable word/phrase or not, the suspect word(s)/phrase(s) and theassigned objectionable/non-objectionable status may be stored inassociation with one another and in association with one or more filterlists, e.g., a filter list included in the filter list database 218 ofFIG. 2. In this way, a subsequent review of a user-created content itemthat contains the previously suspect words(s)/phrase(s) may be assignedan objectionable or non-objectionable status without the need for manualreview. Thus, the system is permitted to become more accurate and moreefficient each time the methods supported thereby are invoked.

In one embodiment, updates of one or more filter lists (e.g., the filterlists stored in association with filter list database 218) may bereceived and automatic re-scrubbing of content items may be performedupon such receipt. In one embodiment, such automatic re-scrubbing may beinitiated for user-created content items meeting a pre-determinedcriterion, e.g., having a creation time following within a certain timeframe, or the like.

The present invention has been described in relation to particularembodiments, which are intended in all respects to be illustrativerather than restrictive. Alternative embodiments will become apparent tothose of ordinary skill in the art to which the present inventionpertains without departing from its scope.

From the foregoing, it will be seen that this invention is one welladapted to attain all the ends and objects set forth above, togetherwith other advantages which are obvious and inherent to the system andmethod. It will be understood that certain features and subcombinationsare of utility and may be employed without reference to other featuresand subcombinations. This is contemplated by and is within the scope ofthe claims.

1. One or more computer-storage media having computer-executableinstructions embodied thereon that, when executed by a processor,perform a method for reviewing content items created by users forpresentation in association with a website, the method comprising:receiving an indication that a content item created by a user has beensubmitted for subsequent presentation at the website, the content itemhaving at least one word or phrase included therein; automaticallyaccessing the content item upon the expiration of a predetermined timeinterval after receiving the indication that the content item is createdby the user; automatically initiating an automatic review processwithout user intervention, the review process comprising: (1) accessinga first filter list that includes at least one word or phrasepredetermined as having an objectionable status associated therewith anda second filter list that includes at least one word or phrasepredetermined as having a suspect status associated therewith, whereinthe objectionable status refers to words or phrases having acategorization indicative of a particular level of abusivenesscomprising at least one of spam, garbage, personal information,profanity, or hate language; and wherein the suspect status refers towords or phrases that are less abusive than the words or the phrasesdetermined to have the objectionable status; (2) assigning an order forcomparison of filter lists within the first filter list and the secondfilter list, wherein the filter lists comprise a global filter list anda website-specific filter list; (3) utilizing the filter lists forcomparison according to the order, wherein the order indicates that whenthe global filter list is accessed, the global filter list is utilizedfor comparison before utilizing the website-specific filter list forcomparison, wherein discrepancies between comparisons to the filterlists are decided in favor of the website-specific filter list, whereinthe global filter list includes a list of words and phrases that areapplicable to content items to be presented on any website that may beaccessed by the user, and wherein the website-specific filter listincludes a list of words and phrases consistent with a website'spublished guidelines that are applicable to content items to bepresented in association with the website; (4) automatically comparingthe at least one word or phrase included in the content item with thefirst filter list and the second filter list, wherein comparingcomprises: (a) utilizing an exact matching technique to determinewhether the at least one word or phrase included in the content itemmatches the at least one word or phrase in the first filter list havingthe objectionable status associated therewith, wherein the objectionablestatus refers to words or phrases having a categorization indicative ofa particular level of abusiveness comprising at least one of spam,garbage, personal information, profanity, or hate language; (b)utilizing a pattern matching technique to determine whether the at leastone word or phrase included in the content item matches the at least oneword or phrase in the second filter list having the suspect statusassociated therewith, and wherein the suspect status refers to words orphrases that are less abusive than the words or the phrases determinedto have the objectionable status; (5) automatically determining whetherthe at least one word or phrase included in the content item matches theat least one word or phrase included in the first filter list or thesecond filter list; (6) when the at least one word or phrase included inthe content item matches the at least one word or phrase in the firstfilter list, assigning an objectionable status to the at least one wordor phrase included in the content item; (7) when the at least one wordor phrase included in the content item is assigned the objectionablestatus, assigning an appropriate categorization to the at least one wordor phrase included in the content item; (8) when the at least one wordor phrase included in the content item is assigned the objectionablestatus, abstaining from subsequently presenting the content item at thewebsite; (9) when the at least one word or phrase included in thecontent item matches the at least one word or phrase in the secondfilter list, assigning a suspect status to the at least one word orphrase included in the content item and marking the content item formanual review by an administrative user; and (10) otherwise, assigning anon-objectionable status to the at least one word or phrase included inthe content item; and at least temporarily, storing the content item,the at least one word or phrase included in the content item, and theassigned status in association with one another.
 2. The computer-storagemedia of claim 1, wherein when it is determined that the at least oneword or phrase included in the content item does not match the at leastone word or phrase included in the first filter list or the secondfilter list, the method further comprises presenting the content item inassociation with the website.
 3. The computer-storage media of claim 1,the method further comprising, based upon the manual review of thecontent item, receiving an indication from the administrative userregarding whether the at least one word or phrase included in thecontent item that is assigned the suspect status is subsequentlyassigned an objectionable status or a non-objectionable status.
 4. Thecomputer-storage media of claim 3, the method further comprising: whensubsequently assigned an objectionable status, based upon the manualreview, storing the at least one word or phrase in association with theassigned objectionable status, thereby updating the status of the atleast one word or phrase as provided on the first filter list and thesecond filter list from a suspect status to an objectionable status; andwhen subsequently assigned an non-objectionable status, based upon themanual review, storing the at least one word or phrase in associationwith the assigned non-objectionable status, thereby removing the atleast one word or phrase from the second filter list.
 5. Thecomputer-storage media of claim 4, the method further comprising, uponupdating the status of the at least one word or phrase as provided onthe first filter list and the second filter list, automaticallyinitiating a re-scrubbing of the at least one word or phrase included inthe content item upon meeting a predetermined criterion.
 6. Thecomputer-storage media of claim 1, wherein the categorization indicativeof a particular level of abusiveness represents both a category and alevel of abusiveness of the at least one word or phrase predetermined ashaving the objectionable status associated therewith, wherein the hatelanguage categorization is associated with a higher level of abusivenessthan the profanity categorization.
 7. The computer-storage media ofclaim 6, wherein when it is determined that a plurality of words orphrases included in the content item matches one of the plurality ofwords or phrases included in the first filter list or the second filterlist, the method further comprises consistently assigning the contentitem as a whole the status of the matching word or phrase of theplurality of words or phrases included in the content item having ahighest level of abusiveness associated therewith.
 8. Thecomputer-storage media of claim 1, further comprising, providing anescalation path to second tier of support upon determining that the atleast one word or phrase included in the content item that is assignedthe suspect status cannot be initially assigned to an objectionable or anon-objectionable status with a high degree of confidence.
 9. A methodfor reviewing content items created by users for presentation inassociation with a website, wherein a computing device is employed toexecute the method, the method comprising: receiving a request from auser at the website for review of a user-created content item submittedfor subsequent presentation at the website; presenting, in associationwith the website, a user interface (UI) configured to receiveinformation input by the user regarding the content item; receiving anindication that the user has completed input of the information at theUI rendered by the computing device; storing the content item inassociation with the information input by the user such that the contentitem is accessible for performing an automatic review process; and uponreceiving the indication, automatically initiating the automatic reviewprocess by comparing at least one word or phrase included in the contentitem with a first filter list and a second filter list, wherein thefirst filter list includes at least one word or phrase predetermined ashaving an objectionable status associated therewith and the secondfilter list includes at least one word or phrase predetermined as havinga suspect status associated therewith, and wherein comparing comprises:(a) assigning an order for comparison of filter lists within the firstfilter list and the second filter list, wherein the filter listscomprise a global filter list and a website-specific filter list; (b)utilizing the filter lists for comparison according to the order,wherein the order indicates that when the global filter list isaccessed, the global filter list is utilized for comparison beforeutilizing the website-specific filter list for comparison, whereindiscrepancies between comparisons to the filter lists are decided infavor of the website-specific filter list, wherein the global filterlist includes a list of words and phrases that are applicable to contentitems to be presented on any website that may be accessed by the user,and wherein the website-specific filter list includes a list of wordsand phrases consistent with a website's published guidelines that areapplicable to content items to be presented in association with thewebsite; (c) utilizing an exact matching technique to determine whetherthe at least one word or phrase included in the content item matches theat least one word or phrase in the first filter list having theobjectionable status associated therewith, wherein the objectionablestatus refers to words or phrases having a categorization indicative ofa particular level of abusiveness comprising at least one of spam,garbage, personal information, profanity, or hate language; (d)utilizing a pattern matching technique to determine whether the at leastone word or phrase included in the content item matches the at least oneword or phrase in the second filter list having the suspect statusassociated therewith, and wherein the suspect status refers to words orphrases that are less abusive than the words or the phrases determinedto have the objectionable status; (e) when the at least one word orphrase included in the content item matches the at least one word orphrase in the first filter list having the objectionable statusassociated therewith, performing the following steps: (i) assigning theobjectionable status to the at least one word or phrase included in thecontent item; (ii) assigning an appropriate categorization to the atleast one word or phrase included in the content item; and (iii)abstaining from subsequently presenting the content item at the website;and (f) when the at least one word or phrase included in the contentitem matches the at least one word or phrase having the suspect statusassociated therewith assigning the suspect status to the at least oneword or phrase included in the content item and marking the content itemfor manual review by an administrative user.
 10. The method of claim 9,wherein the user-created content item is subsequently presented at thewebsite upon receiving an indication that the content item and theinformation input by the user has been reviewed.
 11. One or morecomputer-storage media having computer-executable instructions embodiedthereon that, when executed by a processor, perform a method forreviewing content items created by users for presentation in associationwith a website, the method comprising: accessing a user-created contentitem including at least one word or phrase that is submitted forsubsequent presentation at the website and that is previously determinedto have a suspect status associated therewith, wherein the determinationis performed by executing an automatic review process comprising: (1)accessing a first filter list that includes at least one word or phrasepredetermined as having an objectionable status associated therewith anda second filter list that includes at least one word or phrasepredetermined as having a suspect status associated therewith, whereinthe objectionable status refers to words or phrases having acategorization indicative of a particular level of abusivenesscomprising at least one of spam, garbage, personal information,profanity, or hate language, and wherein the suspect status refers towords or phrases that are less abusive than the words or the phrasesdetermined to have the objectionable status; (a) assigning an order forcomparison of filter lists within the first filter list and the secondfilter list, wherein the filter lists comprise a global filter list anda website-specific filter list; and (b) utilizing the filter lists forcomparison according to the order, wherein the order indicates that whenthe global filter list is accessed, the global filter list is utilizedfor comparison before utilizing the website-specific filter list forcomparison, wherein discrepancies between comparisons to the filterlists are decided in favor of the website-specific filter list, whereinthe global filter list includes a list of words and phrases that areapplicable to content items to be presented on any website that may beaccessed by the user, and wherein the website-specific filter listincludes a list of words and phrases consistent with a website'spublished guidelines that are applicable to content items to bepresented in association with the website; (2) automatically comparingthe at least one word or phrase included in the user-created contentitem with the first filter list and the second filter list, whereincomparing comprises: (a) utilizing an exact matching technique todetermine whether the at least one word or phrase included in thecontent item matches the at least one word or phrase in the first filterlist having the objectionable status associated therewith, wherein theobjectionable status refers to words or phrases having a categorizationindicative of a particular level of abusiveness comprising at least oneof spam, garbage, personal information, profanity, or hate language; (b)utilizing a pattern matching technique to determine whether the at leastone word or phrase included in the content item matches the at least oneword or phrase in the second filter list having the suspect statusassociated therewith, and wherein the suspect status refers to words orphrases that are less abusive than the words or the phrases determinedto have the objectionable status; (c) when the at least one word orphrase included in the content item matches the at least one word orphrase in the first filter list having the objectionable statusassociated therewith, performing the following steps: (i) assigning theobjectionable status to the at least one word or phrase included in thecontent item; (ii) assigning an appropriate categorization to the atleast one word or phrase included in the content item; and (iii)abstaining from subsequently presenting the content item at the website;and (d) when the at least one word or phrase included in the contentitem matches the at least one word or phrase having the suspect statusassociated therewith assigning the suspect status to the at least oneword or phrase included in the content item and marking the content itemfor manual review by an administrative user; and (3) presenting formanual review the user-created content item with the at least one wordor phrase previously determined to have a suspect status; and receivingan indication that the at least one word or phrase previously determinedto have a suspect status has been manually assigned an objectionablestatus or a non-objectionable status; at least temporarily, storing theuser-created content item, the at least one word or phrase included inthe content item, and the manually assigned status in association withone another; and automatically updating the second filter list byremoving the at least one word or phrase previously determined to have asuspect status association therewith.
 12. The computer-storage media ofclaim 11, the method further comprising, when an indication is receivedthat the at least one word or phrase previously determined to have asuspect status has been assigned a non-objectionable status, presentingthe user-created content item in association with the website.
 13. Thecomputer-storage media of claim 11, the method further comprising, whenan indication is received that the at least one word or phrasepreviously determined to have a suspect status has been assigned anobjectionable status, refraining from presenting the user-createdcontent item in association with the website.
 14. The computer-storagemedia of claim 11, wherein automatically updating the second filter listby removing the at least one word or phrase comprises automaticallyinitiating a rescrub of the content items with the updated second filterlist.
 15. The computer-storage media of claim 11, wherein presenting formanual review the user-created content item with the at least one wordor phrase singled out from a remainder of content in the user-createdcontent item comprises rendering a user interface (UI) that includes apresentable menu of options for changing the status of the at least oneword or phrase, previously determined to have a suspect status, to anobjectionable status or a non-objectionable status, and wherein the atleast one word or phrase previously determined to have a suspect statusis singled out from a remainder of content in the user-created contentitem by having a suspect status indicator displayed parentheticallyadjacent to the at least one word or phrase.
 16. The computer-storagemedia of claim 11, wherein, when the content items created by the usersfor presentation in association with the website have already beenpublished thereon, the at least one word or phrase previously determinedto have a suspect status is linked to a violation report submitted by avisitor to the website for review, and wherein the linked violationreport is accessible at a user interface (UI) that includes apresentable menu of options for changing the status of the at least oneword or phrase, previously determined to have a suspect status, to anobjectionable status or a non-objectionable status.